arXiv: 1508.01321 vl [cs.CL] 6 Aug 2015 


On Gobbledygook and Mood of the Philippine Senate: 
An Exploratory Study on the Readability and Sentiment 
of Selected Philippine Senators' Microposts 


Fatima M. Moncada and Jaderick P. Pabico 
Institute of Computer Science 
University of the Philippines Los Banos 


Abstract 

This paper presents the findings of a readability assessment and sentiment 
analysis of selected six Philippine senators’ microposts over the popular Twitter 
microblog. Using the Simple Measure of Gobbledygook (SMOG), tweets of Sena¬ 
tors Cayetano, Defensor-Santiago, Pangilinan, Marcos, Guingona, and Escudero 
were assessed. A sentiment analysis was also done to determine the polarity of 
the senators’ respective microposts. Results showed that on the average, the six 
senators are tweeting at an eight to ten SMOG level. This means that, at least 
a sixth grader will be able to understand the senators’ tweets. Moreover, their 
tweets are mostly neutral and their sentiments vary in unison at some period of 
time. This could mean that a senator’s tweet sentiment is affected by specific 
Philippine-based events. 


1. Introduction 


Readability refers to a certain c lass of peop l e’s pe rception of a text’s compellingness 
and requisite comprehensibility ( McLaughlirJ . 1968). The degree of a text’s compelling¬ 
ness can be measured by determining the proportion of a certain class of people who 


read the text by choice. People wh o belon) 
similar Terminal Educational Age ( Abram: 


to the same class are those with closely 
A person is compelled to a text T 


ng to tin 
I I 19631) . 


she is reading if she understands T. Therefore, comprehensibility is a requisite to 
compellingness. A text’s measure of r eada bility is ultimately dependent on a text’s 
linguistic characteristics ( McLaughlin! . 1968 ). Texts which are full of medical jargon, 
for example, have high readability scores among medical doctors but might have low 
readability scores among lawyers. This might have been one of the reasons why senator- 
judge Rodolfo G. Biazon of the Philippine Senate’s 11th Congress, a trained military 
man being the former Chief of Staff of the Armed Forces of the Philippines, stated the 
following during the trial of the impeachment case against former Philippine President 
Joseph Ejercito Estrada: 


... because of the legal exchanges that I could hardly put together, I said 
let’s do away with this legal gobbledygook... 


This seemingly sarcarstic retort, once a favorite topic of conversation in the news media 
and among personalities in the broadcast version, was the result of a series of exchanges 
of lengthly legalese arguments among lawyers for and against President Estrada. The 
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senator’s witty statement was meant to remove “legalistic gobbledygook” from “intel¬ 
ligent communication” within the Senate chamber that is composed of members who 
were nationally elected to represent, not only certain classes of people, but all Filipino 
people. 


Given a text T, how can one put a value on the readability of T? Word and sen¬ 
tence lengths are t he ling uist ic characteristics that best predict a text’s reading diffi¬ 
culty ( McLaughlinl . 19691 1968 1 or comprehensibility. Thus, by measuring word and 
sentence lengths, one can determine a text’s readability for a certai n class of people, 
given that the text has been proven to be compelling for them (McLaughlin, 1968 1. Not 
until the use of legal gobbledygook was minimized in the proceedings of the Estrada 
impeachment that the Filipino people, already compelled to follow the proceedings 
because of the apparent human drama involving the highest official of the land, was 
able to totally comprehend it. 


1.1. Simplified Measure of Gobbledygook 


The Simple Measure of Gobbledygook or SMOG is a readability formula which cal¬ 
culates the approximate number of years of education required for a person to com¬ 
prehend a given text. This formula is called the SMOG Grade, a function di rectl y 
proportional to the total number of polysyllabic words in a text ( McLaughlinl . Il968h . 
In computing for the SMOG Grade, a total of 30 sentences must be sampled from the 
text: ten consecutive sentences at the beginning, ten at the middle, and ten at the 
end. All words with more than two syllables must be counted to get the total number 
of polysyllables. Abbreviated words mu st be read as unabbreviated and numerical 
characters must be spelled out as well ( McLaughlinl . 1969 ,1. The SMOG Grade 7 (T) 
of a text T is shown as Equation [TJ where 4> is the number of polysyllabic words in T, 
and a is the number of sentences in T: 


7 (T) = 1.043 x 



3.1291 


(1) 


It was experimentally found out that the SMOG pre cise formul a yields a correlation 
coefficient of 0.985 and a standard error of 1.5159 (McLaughlin, Il969h . From the 
precise formula given above, a simpler equation ip(T) can be derived (Equation [2]) : 


m = 30 + 


( 2 ) 


Although the simpl ified SMOG formula is less accurate 
dally in fieldworks ( McLaughlin . 1 1969 1. 


it is more preferred espe- 
Its simple implementation and speed of use 
while still providing a rigorous method of m easu ring re adab ility are what compelled 
researchers to use it instead of Equation [Tl ( McLaughlinl . 1969l h 


1.2. SMOG in Readability Assessment of Health Messages 


Because of SMOG’s usefulness in accessing communication materials from a certain 
class of experts to a more general class of people, it has since been used in dif¬ 
ferent scientific researc hes that aim to assess th e readability of diffe re nt health com- 
munication materials ( Ache and Wallace . 2012 : Auta et al. . , l2011af bl: iBrandt et al 
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2005 : Gill et al. . 2012 : Svider et al. . 2013 : Wallace et al. . 2010t Wilson . 2009h and 
other health-related documents ( Covne et all ' 20031) . While most researches deal with 
printed communication materials, some s tudies assessed the readability of various 


health-related information available online (ICherla et a.l.l. [ 2012 : Edmunds et al 
Honod . l201flt Stossel et all 20121 : Walsh and Volskol . 1200 


2014 


In general, there are two approaches used in readability assessment using SMOG. First, 
the SMOG Grade is used to determine which grade level will be able to understand 
various educa tional and communication materials that are targeted towards a gen¬ 
eral a udience ( Auta et all 2011atlBrandt et all . 1200.41(1111 et al. . 2012 : Makoskv et al. . 
2009). Another approach used in readability researches is zeroing in to a particular 


audience group and determining, through the use of the SMOG formula, whether the 
communication materials can be understood easily by the specified target audience. 

The latter approach is more commonly u sed in researches that aim to d esign , 
develop, test, and modify health message s dHolt et, al. 


201 0: Makoskv et al. . 120091 : 


Neuhauser et al. . 20091 : IPatel and Simpsonl . |2010|) or when the communicatio n mate- 


rial being assessed was originally designed for a specific target audience ((Swartz, 
201 0). For example, in developing printed educational m aterials o n prostate cancer 


for church-attending African-American men, Holt et al (Holt et ah, 1201Q ) used the 


SMOG formula to assess their original materials and then revise them to a desirable 
level of sixth-grade reading difficulty. On the other hand, Swartz ( Swartzl . 2010 ) 
examined the readability of handouts and brochures on pediatric otitis media targeted 
towards parents. He determined whether the obtained SMOG Grade of eight corre¬ 
sponds to the reading capability of the publications’ intended audience. In addition, 
he also explored the correlation between the SMOG Grade and the parents’ actual 
reading satisfaction. 

Aside from readability assessments of different health messages, SMOG has also been 
used as a tool in determining the effectiveness of semantic and syntactic text simplifi¬ 
cation. Nowadays, text simplification is done automatically through natural language 
processing, specifically using synonym generation and explanation generation. In ana¬ 
lyzing whether the simplified text is indeed more readable than the original text, 
the SMOG formula is often used. If a simplified text scored lower than the original 
text in terms of SMOG, then the automated tex t simplification is considered effec¬ 
tive ( Kandula et al. . 20ld) . However, Leroy et al. ( Lerov et al. . 2013 ) noted that text 
simplification based on SMOG and other readability tests often results to more dif¬ 
ficult text because these readability tests are focused on the writing style (i.e. word 
and sentence length) rather than the content itself. Therefore, it is of key importance 
that when using SMOG, careful interpretation and/or conclusions must be made in 
accordance to the tool’s limitations. 


1.3. SMOG and Other Readability Tests 

SMOG is often used in combination with other readability tests such as the Flesch- 
Kincaid Grade Level, Flesch Reading Ease, Fry Readability Formula, and the Gun¬ 
ning Fog Index. For example, Gill et aLcitep7 assessed the readability of publications 
released by the United States Center for Disease Control and Prevention on concussion 
and traumatic brain injury using SMOG and three other different readability tests. 
The materials’ Gunning Fog Index and Flesch-Kincaid Grade Level varied very closely 
at 11.1 and 11.3 respectively, with a Flesch Reading Ease index of 49.5. Interestingly, 
the computed SMOG grade for the tested materials was 12.8, notably higher than the 
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two other tests. Another study which assessed the readability of patient education 
materials produced for the low-income population of the United States yielded similar 
results, where the SM OG grade (9.89) was significantly higher than the Flescli-Kincaid 
Grade (7.01) (IWilsonl . 2009). Consistent to such pat tern, readability assessment of dif¬ 
feren t medicine information in two separate studies ( Auta et all 201 lbl : Wallace et, all 
2010t ) resulted to SMOG Grades that were higher than the Flesch-Kincaid Grade by 1 
to 3 levels. In studies where the objects o f readability assessment are I nternet-based 
or online health information (lUherla et all 120121 : lEdmunds et all 12014 IStossel et all 
20121 : IWalsh and Volsko , 20081) . the SMOG Grades remained significantly higher than 
their corresponding Flesch-Kincaid Grades; while, the Gunning FOG indexes are either 
equal to or slightly higher than the SMOG Grade. 

When the SMOG formula is used with other readability tests and the results vary, 
some researchers would interpret the results collectively. For example, in assessing 
the reada bility of online resources on Graves’ disease and thyroid-associated ophthal¬ 
mopathy ( Edmunds et all 12014 1. the US Department of Human and Health Sciences 
(USDHHS) standards for reading difficulty was used in interpreting the varying indexes 
obtained for SMOG, Flesch-Kincaid, Gunning-Fog, and Flesch Reading Ease. Fol¬ 
lowing the recommended readability level (4 to 6) for online materials by the USDHHS, 
the study concluded that the online resources that were analyzed are too difficult for 
its audience to understand, with readability indixes of 11 for the Flesch-Kincaid for¬ 
mula, 13 for the SMOG and the Gunning-Fog formula, and 46 for the Flesch Reading 
Ease formula. 

On the other hand, the Centers for Medicare and Medicaid Services or CMS recom¬ 
mends the use of S MOG as a stand ard in making written materials effective and clear 
to its audience ( Stossel et all 12012 1. Hence, other researchers opt to consider just the 
SMOG results when significant differences among the readability indexes are encoun¬ 
tered. For example, considering t he SMOG formula as the gold standard of measuring 
readability, Fitzsimmons et al. ( Fitzsimmons et al. . 2010f) interpreted the difference 
between the SMOG Grade and the Flesch-Kincaid Grade Level as the latter’s under¬ 
estimation of a text’s readability. In their research, they were able to determine that 
the Flesch-Kincaid formula resulted to a mean underestimation of 2.52 grades in deter¬ 
mining the readability of online information on Parkinson’s disease. Hence, to avoid 
underestimation, they suggested that the SMOG formula should be generally preferred 
when assessing online health information. 


1.4. SMOG, Twitter, and Integrating Readability 


TIME Magazine has recently created a web applicat ion for deter mining how smart a 
given tweet is by computing for its SMOG Grade ( TIMEl . l2015af bh. Using the web 
application, TIME has named the top 50 smartest celebrities on Twitter by a nalyzing 
the 50 0 most followed twitter users’ tweets and comparing their SMOG Grades (' TIME '. 
2015al) . Similarly, 1 million tweets were analyzed using the SMOG formula and resu lts 
showed that 33% of the sampled tweets are only at the fourth grade level ((TIME . 
2015b ). TIME has argued that the 140-character limit to a tweet makes it difficult, 
but not impossible, for a Twitter user to compose a tweet that has a high SMOG grade. 
And while the findings of the analysis showed that politicians are the ones who tend to 
tweet using polysyllables, the results should not be t reated as conclusive since the study 
did not follow proper sampling techniques ( Steel and Torrid 1980l) . Nevertheless, the 
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potential use of SMOG in assessing the readability of tweets is highlighted in TIME’S 
study. 


Additionally, while SMOG is tail ored f or long er texts, a SMOG formula for short texts 
has already been developed (lUniversitvI . In.d.il . Hence, it is deemed appropriate to use 
SMOG in analyzing the readability of tweets which are, by nature, short texts. The 
SMOG formula i/j s for short texts is given in Equation [3] below: 


4> s =3 + 


-(30 —cr) + , 
a 


(3) 


However, up to date, the actual use of SMOG in assessing the read a bility of tweets has 
not been exhaustedly studied. Although, Guo et al. ( Guo et all 1201 11 have already 
suggested integrating readability on the Twitter search engine by embedding the read¬ 
ability scores into the search results using the following steps: 


1. Accessing Twitter; 

2. Requesting for a Twitter archive; 

3. Parsing tweets; 

4. Computing Readability; 

5. Embedding scores into the search results. 


Integrating readability in Twitter can potentially_enhance the retrieval of relevant 
data for academic and/or commercial purposes ( Guo et al. . 201ll h especially now that 
data mining has become the subject of numerous scientific studies and market research. 
But the reliability and effectiveness of the readability assessment must be the foremost 
consideration; hence, it is of utmost importance that the readability tests and tools 
that will be used in any assessment are context-based and yields valid and reliable 
results. 


2. Sentiment Analysis on Twitter 


Nowadays, sentiment analysis is often use d as an o pinion-mining too l in various social 
media platforms, especially in Twitter ( Pak and ParoubekL 2010). Modelling the 


public’s mood and certain mapping socio-economic phenomena (Bollen et al., 20111 


are also some of the rationale behind the plethora of sentiment analysis researches 
involving the high-traffic social media platform. 

A number of sentiment analysis approaches had al ready been em ployed in the past. 


Some of which are p hrase-level sentiment polarity (Wilson et al., [2005 1 and semantic 


orientation (Turney, 2002ll . Mor eover, in doing sentiment analysis of tweets, a Naive 
Bayes classifier is often utilized (Go et, all 120091 : 1.Tose et ah . 2010h . While the three- 
way clas sificatio n (Go_ et ah , 200911 has become popular over the course of time, some 


studies ( Bollen et all 201 ill implement psychometric instruments to classify words 


into, not only three, but six moods or sentiments. 


For the purpose of simplification and appropriating our methodology with the length 
of texts under study (tweets), we used a Naive Bayes classifier for sentiment analysis. 
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3. Methodology 


Twitter provided an Application Progra mming: Interfa ce (API) to allow for the auto¬ 
matic “scraping” of Twitter microposts ( Dorsev et al. . 200(f). Scraping is the process 
of extracting pertinent data from web pages obtained from “crawling” the Internet. 
Crawling a set W n of n web pages W n = {po,Pi, ■ ■ ■ ,Pni} means downloading the 
subset W „i = {pi,P2, ■ ■ ■ ,Pni } C W n web pages given the initial web page pq. From 
the respective uniform resource locator (URL) links in hypertext markup language 
(HTML) anchor tags found in a web page pi, the next web page Pi+i can be obtained 
and whose respective data can be scraped, Mpi,p i+ i G W n . The Twitter API is a 
set of computer commands provided by the Twitter developers for exclusive use of 
programmers to allow them to tap into the T witter data stream and gather tweets at 
a specific timeframe and geo-location ( Dorsev et al. . 20061) . Once the streamed tweets 


have been collected by a computer program Co that uses the API, they will become 
inputs I to two automated classifiers C\ and C 2 which will respectively output the 
SMOG grade and contextual sentiment polarity of the tweets (Figure [l]). 



Figure 1: The functional relationships between the Twitter and its API, and the com¬ 
puter programs Co, C\, and C 2 developed in this research to estimate the 
SMOG grade and sentiment of a tweet. Arrows mean the direction of data 
or request for data, while boxes means computer programs or commands. 


Using Twitter API vl.l in Co, the tweets of six Philippine senators, whose Twitter 
accounts were listed as verified by the Official Gazette, were collected. All tweets 
from August 15, 2013 to August 15, 2014 of Pia Cayetano, Miriam Defensor-Santiago, 
Chiz Escudero, Kiko Pangilinan, TG Guingona, and Bongbong Marcos were processed 
by Cq to become separate inputs I to C± and C 2 . 


Building on the PHP class called Text Statistics developed by Child (Child, n.d.l ). the 
classifier C\ that calculates the SMOG Grade of short texts was developed. Corrections 
to appropriately compute for the readability of short texts (i.e., tweets) were made on 
the original computer code by Child. 


For the sentiment analysis C 2 , the tweets’ polarities were identified using a Naive 
Bayesian classifier that classifies a given word’s sentiment as positive, negative, or 
neutral. Several unambiguous English and Filipino words were collated, assigned with 
a polarity classification, and used as library for the sentiment analysis. 
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4. Results and Discussion 


4.1. SMOG Readability of Senatorial Tweets 

All Twitter accounts of the six senators showed a high level of SMOG readability. 
Their respective average SMOG Grades are shown in Figure [2] 

Senator Name 

Cayetano 

Defensor- Santiago 
Escudero 
Guingona 
Marcos 
Pangilinan 

“I-1-1-1-1-1-1-1-1-1-1 

8.0 8.2 8.4 8.6 8.8 9.0 92 9.4 9.6 9.8 10.0 

Average SMOG Grade 

Figure 2: The average SMOG Grade of each senator over the observation period. 

The lowest average SMOG Grade computed was 8.64 (Marcos). The highest, 9.22, 
has a small margin of difference than the rest of the computed values: 9.11, 9.15, 9.17, 
9.18. This means that on the average, the senators’ tweets will be most comprehensible 
to those who have already completed eight to ten years of formal education. In the 
newly-implemented Philippine educational system (K-12), that is equivalent to late 
elementary school to early junior high school. 


Lady Senators 


4.2. Time-dependent SMOG Readability Assessment 

To find out whether the SMOG Grades of the six senators’ tweets shift through time, 
the computed SMOG Grades were averaged per month and are presented as Figure[3l 

The SMOG Grade trend of Cayetano, Escudero, Guingona, Marcos, Pangilinan, and 
Defensor-Santiago varied closely. This means that the style employed by the senators 
in writing their posts do not vary that much and rarely shift over time. Although, 
Cayetano’s and Defensor-Santiago’s Twitter accounts showed a significant downward 
shift in readability around August-September 2013 and February-June 2014, respec¬ 
tively. 


4.3. Sentiment Analysis of Senatorial Tweets 

Results of the sentiment analysis showed that most of the senators’ tweets are neu¬ 
tral, otherwise positive. A breakdown of the dominant sentiment for each senator is 
presented in Figure |U 
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2013--2014 


Inclusive date of tweet collection 


Figure 3: The monthly average SMOG Grade of each senator showing their respective 
trends over the 13-month observation period. 


Among all senators, only Cayetano tweets mostly positive messages. Marcos, on the 
other hand, tweets positive and neutral messages equally. Moreover, analysis of his 
tweets revealed that the senator virtually does not tweet negatively. 

Analysis of the tweets’ sentiment vis-a-vis the senator’s gender revealed that more male 
senators tweet neutral tweets and that female senators tend to tweet both positively 
and neutrally (Figure [5]). 

4.4. Time-dependent Sentiment Analysis 

Each senator’s tweets per month were analyzed and results showed that the sentiment 
of some senators’ tweets vary in unison (Figure [6]). 

Cayetano, Pangilinan, Guingona, and Defensor-Santiago’s tweets went from positive 
to neutral around November 2013, the month of All Soul’s Day celebration, and went 
back to positive around December 2013 to January 2014, usually a time of celebration 
for Filipinos due to Chirstmas and New Year’s Day. 

Moreover, the sentiments of Guingona, Pangilinan, and Defensor-Santiago’s shifted 
downward, although in different slopes, around February 2014 and stayed neutral 
until around July 2014. It is also of particular interest to note that Defensor-Santiago’s 
tweets around February 2014 are mostly negative. 

Cayetano and Marcos’ tweets, on the other hand, shifted from neutral to positive 
around the month of May of 2014 and noticeably went down around the month of July, 
when Guingona, Defensor-Santiago, and Pangilinan’s tweet sentiments went up. 


5. Conclusions 

Our findings showed that Senators Marcos, Escudero, Pangilinan, Defensor-Santiago, 
Cayetano, and Guingona’s tweets are, on the average, between a SMOG Grade of eight 
to ten. Moreover, a time-dependent analysis revealed that the SMOG Grades of the 
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Senator Name 


Pangilinan 


Sentiment 

i Negative 
■ Neutral 
I Positive 



100 200 300 400 500 600 700 800 900 

Number of tweets 


Figure 4: The total number of tweets for each senator according to sentiment polarity. 



Figure 5: The average number of tweets by gender according to sentiment polarity. 


senators’ tweets do not vary that much over time. This means that, the audiences who 
would understand a senator’s tweets are those who have attained at least the sixth 
grade level of education (if preparatory school is considered). 

Social media users nowadays are largely composed of audience groups around that 
age and level of education. Hence, we deem the eight-to-ten range of SMOG Grade 
appropriate to the potential, if not prospective, audiences of the senators. However, 
ours being an exploratory study, we recommend that a more extensive research with 
a larger data set be done to increase the validity of such conclusion. Nevertheless, if 
the senators would like to expand the reach of their social media following, especially 
in Twitter, a SMOG Grade range of eight to ten may prove to be narrow than what 
would otherwise help achieve such goal. 

On the other hand, sentiment analysis of the senators’ tweets revealed that most 
of them post neutral messages and positive, otherwise. Although five out of the six 
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- Cayetano 

— Defensor-Santiago 

— Escudero 

— Guingona 
Marcos 



Inclusive date of tweet collection 

Figure 6: Monthly dominant sentiment tweets of the senators. 


senatorial Twitter accounts that were assessed revealed a few negative sentiments. This 
could mean that the senators, being public figures, rarely posts negative messages as 
a form of cautious act. Moreover, most of these senators do not personally handle 
their Twitter accounts and it is their communications staff who actually post on their 
behalf; therefore, the neutral or positive posts could very well be considered as a digital 
online presence effort rather than public communication per se. 

Furthermore, the fact that some of the senators’ tweet sentiments vary in unison during 
particular periods of time could mean that events, be it political or not, potentially 
affect the messages’ sentiment. For example, four senators tweeted neutral messages 
during November 2013 and tweeted positively from December 2013 to January 2014. 
Coincidentally, Filipinos are known to be very appreciative of the Christmas and New 
Year’s season which could be one explanation why the senators’ tweets shifted from 
neutral to positive during those periods. However, to be able to correlate these two 
variables scientifically, it is suggested that succeeding studies make use of larger data 
sets and a more extensive sentiment analysis tool. 
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